Perception of quality of care using patient reported experience measures (PREMs) in a cohort of adults with HIV: A cross-sectional study

Human immunodeficiency virus (HIV) infection is considered a chronic disease. Antiretroviral therapy has allowed persons with HIV (PLWHIV) to achieve the 90-90-90 objectives proposed by the World Health Organization for 2020; but an additional challenge is getting an adequate health-related quality of life. A determining factor in the health-related quality of life of PLWHIV is the health care they perceive to receive. In this sense, we aimed to assess the perception of the outpatient care provided and to identify possible areas for improvement in a single-center, cross-sectional study at the HIV unit of Hospital Clínic, Barcelona. We sought patient reported experience measures by an anonymous e-survey with 11 statements based on a 1 to 6 Likert scale, and a final question measuring user satisfaction and loyalty through the Net Promoter Score (NPS). All PLWHIV with at least a clinical visit between January 1, 2020 and October 14, 2021 were invited. Of 5493 PLWHIV e-mailed, 1633 (30%) responded to the survey. The overall evaluation of clinical care was very favorable. The evaluation of the physical environment and facilities and the time spent in the waiting room received the lowest scores. According to the Net Promoter Score test results, 66% of respondents were willing to recommend this service, and 11% were not. Thus, monitoring patient reported experience measures in PLWHIV actively receiving outpatient care in our hospital allowed to identify the users’ perception on quality of the care received, to determine the rate of satisfaction with the care, and to identify areas for improvement.


Introduction
"Health-related quality of life" [1] (HRQoL) is defined as the perception that a patient has regarding the effects of a disease or the application of a certain treatment in areas of his or her life, especially on his or her physical, emotional and social well-being.
In the last century, the substantial development of new pharmacological products and health technology has increased the interest on the measurement of quality of life in health care.
Traditional medical outcome variables such as clinical symptoms, survival rate, and safety and clinical efficacy of a drug are no longer sufficient to adequately reflect the effect of This work is part of a study that evaluates the optimal model of care for HIV patients (VIIV V02-2020 grant). "Programa coordinado de intervención intersectorial e interdisciplina enfocado a la prevención combinada del VIH y a la atención a la cronicidad en las personas con VIH, con el fin de potenciar el continuum de los cuidados a lo largo de todo el proceso asistencial (A coordinated program of intersectoral and interdisciplinary intervention focused on the combined prevention of HIV and care for chronicity in people with HIV to enhance the continuum of care throughout the entire care process." JMM received a personal 80:20 research grant from Institut d'Investigacions Biomèdiques August Pi i Sunyer (IDIBAPS), Barcelona, Spain, during 2017-23. The authors have no conflicts of interest to disclose.
The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request.
The study was approved by the HCB Ethics Committee. Before starting the survey an explanation and "Data policy" was included in the email as an informed consent. Patients who agreed with these terms completed the e-survey. health care and interventions, and it is considered necessary to analyze the perspective of the person with a disease too. What matters is how the person with a disease feels rather than how health professionals believe they should feel based on clinical measures. [2] In that way, whether therapy can achieve a life worthy of being lived, both from the social and psychological perspective as well as the physical perspective, should be assessed.
HRQoL has become an important measure of the impact of medical care. Currently, the focus is on the quality, rather than the quantity, of life. Gradually, the patient experience has been incorporated into analyses of care processes to evaluate quality from the perspective of the person receiving the service [3] to improve it. For this, new assessment tools have been introduced: PROMs (patient-reported outcome measures), related to treatment objectives, and PREMs (patient-reported experience measures), related to the experience of a treatment or intervention. [4][5][6] The management of human immunodeficiency virus (HIV) infection has been changing over the last decades, adapting to the needs of persons living with HIV (PLWHIV) and, above all, influenced by the available therapeutic arsenal. Currently, the high efficacy of antiretrovirals, with the consequent sustained viral suppression and disappearance of opportunistic diseases, has drastically reduced the mortality associated with HIV [7][8][9] in way such that PLWHIV show a life expectancy similar to that of the general population, turning HIV infection into a chronic disease. [10] In addition to the goals outlined by the World Health Organization to control HIV infection for 2020, [11] that is, 90% of PLWHIV diagnosed, 90% treated and 90% have undetectable viral load, a fourth objective focused to achieve a correct quality of life in 90% of these patients has been also raised in recent years. [12] The current challenge of the outpatient units caring for PLWHIV is to redefine what the optimal model of care should be, how to adapt resources according to the needs of each patient and how to incorporate this fourth 90% goal related to quality of life.
In recent years, tools have been introduced for the collection of patient-reported outcome measures in PLWHIV within clinical trials and controlled studies [13][14][15][16][17] to assess the possible benefits and risks of an intervention more accurately; however, the information reported in the literature on PREMs in PLWHIV is scarce. [18,19] In the HIV Unit of the Hospital Clinic of Barcelona, nearly 6000 PLWHIV are currently receiving outpatient care, and health workers at the Unit have been committed for years to updating and improving the approach and care of them. In addition to assess the quality of care with well-established indicators of safety and efficacy, [20] we aimed to know about the experience of PLWHIV, which some authors consider the third pillar in the evaluation of quality of care and a key element for improving health care in the context of person-centered care. [21] The objective of this study was to assess the user perception on quality of the care received, to determine the rate of satisfaction with the care and to identify potential areas for improvement.

Methods
This was a cross-sectional study that evaluated through an anonymous electronic survey (e-survey) the perceived satisfaction of PLWHIV with the care received in the HIV Unit of the Hospital Clinic of Barcelona. All patients for whom we had records for at least 1 visit between January 1, 2020 and October 14, 2021 were considered candidates to respond the survey. Two analysis instruments were used, a PREM assessment survey and the net promoter score (NPS).

Instruments
The survey was prepared with the help of the Patient Care Research Team at our hospital, taking as a model similar surveys carried out in other areas of care for chronic patients of the Hospital Clinic of Barcelona and the recommendations of the Agency for Quality and Health Assessment of Catalonia. [22] The questionnaire includes 5 questions regarding demographic information to determine the profile of the respondent and 11 statements scored by the respondent based on the degree of disagreement or agreement using a Likert scale ranging from 1 to 6. These statements are as follows: Q1 The location where the consultation occurred was appropriate. Q2 The time spent in the waiting room was acceptable. Q3 The information received from the professionals who treated me was clear and concise. Q4 The vocabulary used was adequate. Q5 The information received during the visit responded to my needs. Q6 I believe my opinion mattered, and I felt like a participant in decision-making about my health. Q7 The visit was carried out in a climate of trust that allowed me to express my concerns. Q8 The time allocated to doubts and/or questions was sufficient. Q9 The total length of the consultation seemed to be the right amount. Q10 I feel that the attention received during the visit was adequate. Q11 The time spent in the consultation was worth it (considering travel, waiting time, etc).
In addition to the survey, a final question, "How likely is that you would recommend this Service to a friend or colleague?," based on a 0 to 10 scale was associated with measuring user satisfaction and loyalty with our service through the NPS indicator. [23] The electronic questionnaire was implemented in 3 languages: English, Spanish and Catalan, using the online survey tool LimeSurvey hosted at Hospital Clinic of Barcelona.
The system was programmed to mass-send the survey to the registered email of all PLWHIV for whom there were Table 1 Characteristics of the population who responded to the survey.

Variable
Summary statistics clinical records for at least 1 visit between the stablished period, January 1, 2020 and October 14, 2021. Seven and twelve days after the initial email on 10/14/2021, the system automatically sent 2 reminder emails to PLWHIV who had not yet responded.

Statistical analysis
Subjects were classified into 3 groups based on the scores obtained from the NPS question: those scoring 9 or 10 are considered Promoters (they would recommend this service to a friend or family member); those scoring 7 to 8, Passives, and those scoring 0 to 6 Detractors. The final NPS score was calculated as the difference between the percentage of Promoters and Detractors. Qualitative characteristics were described using absolute frequencies and percentages and were compared among the NPS groups with the chi-square test. Quantitative variables were summarized as the mean and standard deviation (SD), and comparisons among the NPS groups were conducted using analysis of variance. The pattern of relationships among the responses to the questionnaire was analyzed using the multiple/joint correspondence analysis (MCA) with 2 dimensions to capture the Guttman effect that affects preference data and occurs when higher order dimensions are polynomial functions of the first one. In our data the arch shaped form suggests a non-symmetric quadratic function. In the MCA graph the first dimension defines a scale of scores from worst to best, from left to right, and the second defines a contrast between extreme responses (positive part of the axis) and neutral/moderate responses (negative part of the axis

Results
The e-survey was submitted to 5493 PLWHIV. Figure 1A shows the number of surveys received daily. We collected 1633 (30%) correctly completed surveys, and 208 (4%) respondents only partially completed the questionnaire. One hundred and 7 (2%) PLWHIV accessed the e-survey invitation but did not participate. Figure 1B shows the flowchart of the surveys. Sixty-eight percent (n = 1250) of the surveys were answered in Catalan, 30% (n = 559) in Spanish and 2% in English (n = 32). Most PLWHIV who responded were older than 35 years (90%) and the respondents were evenly distributed among the 3 age groups outlined in the questionnaire. Fifty-eight percent had received higher education, and only 8% reported primary education as their highest level. Two-thirds of the respondents were of Spanish origin, 10% were from other European countries and 21% were from Latin America. Sixty percent had a long-standing HIV infection diagnosed more than a decade ago. Baseline characteristics of the population are shown in Table 1. The age distribution for the cohort of responders was roughly similar to that of the whole cohort; the percentage older than 45 years was 63% and 54%, respectively. The percentage of PLWHIV with higher education (58% vs 46%) and of Spanish origin (66% vs 53%) was higher in the cohort of responders than in the whole cohort.
Overall, the average score for all the statements in the survey was above 4; that is, the responses were favorable, except that for question Q2 related to the time spent in the waiting room that was the worst-rated statement with an average score of 3.9 (SD 1.5) and viewed unfavorably (≤3) by 36% of the users (see Fig. 2A and Table 2). Question Q1, which assesses the physical environment and facilities, received an average score of 4.4 (SD 1.6), and 27% of users felt that the room for consultation was inadequate. Questions Q3, Q4, and Q5, which address how medical information is delivered and received, were very well rated, with average scores ≥ 5.4 (SD 1.2). The clarity of the information received from health care professionals as well as the language used were considered adequate (≥4) for more than 93% of respondents; in fact, Q4 was the highest scoring statement among those in the questionnaire. Finally, 93% of the respondents indicated that the information received responded to their needs. Question Q6, analyzes how the user perceived their involvement in decision making and 91% of the participants indicated that it was adequate. The following group of questions, from Q7 to Q10, delve into the assessment of the quality of the visit: Q7 assesses the development of a climate of trust in which concerns can be expressed, Q8 and Q9 assess time allocated for questions and total visit length and Q10 assess attention during the visit. All the responses in this area were very positive, with 90% of respondents agreeing with the survey statement. For the last question, Q11, which assesses overall experience, the average score was 4.9 (SD 1.4) and 84% of the users responded that the time spent in the consultation was worth it, taking into considering travel, time and waiting.
The average NPS was 8.7 (SD 1.9). Sixty-six percent of the users were classified as promoters. Only 11% scored below 7 and, therefore, were classified as detractors. Thus, the net value of the resulting NPS was 55 (see Fig. 2B). For the NPS results, we did not assess differences by age group or education level, but clearly, Spanish respondents were less likely than respondents of any other origin to be classified as promoters (P = .0005).
In the graph of the MCA (Fig. 3), respondents who were considered promoters were associated with the maximum score, 6, for most questions, given their closeness to these scores. Respondents who were considered passives differed slightly from the previous group, being closer to scores of 4 and 5. The detractors, are further from the center, in the negative part of the axis, because they represented the group who provided the lowest scores.
The groupings or clusters of scores indicated that there was a pattern in these responses: those who responded with a 1 to 1 of the statements tended to do the same to other statements (except for statements about the location of the consultation (Q1), waiting time (Q2) and time spent in total (Q11)), as indicated by the cluster at the top left. The same occurred with the cluster for the score of 6, for which there was a grouping for all statements. The other scores were more dispersed, indicating more variation in the responses.

Discussion
The availability and efficacy of antiretroviral treatment have allowed to achieve the goals proposed by the World Health Organization for controlling HIV infection by 2020 [11] in our cohort. Specifically, of the 5493 PLWHIV followed in the HIV Unit of the Clinic Hospital of Barcelona, 99% were receiving antiretroviral treatment, and 94% had an undetectable viral load during the study period.  In accordance with the redefinition of therapeutic success proposed by the scientific community, [12] which includes quality of life as a fourth goal to be achieved in 90% of PLHIV, we analyzed the quality of care we provide in our unit by reviewing the model of care offered to users and incorporating patient perception through an e-survey, with the ultimate goal of improving the service.
First, the response rate to the online invitation to participate in the study was remarkably high considering that PLWHIV had not been previously informed, nor the project was publicly advertised. One-third of the candidates answered our request, a percentage very similar to that obtained in an electronic format survey conducted in a different therapeutic area of the hospital. [24] These data suggest that PLWHIV are sensitive to collaborating in a project that can help improve their care.
The demographic profile of the people who responded to the survey differed slightly from the average profile of PLWHIV of the unit: higher percentage of people of Spanish origin and higher education among the survey respondents. These subtle    differences could indicate a selection bias about the persons included in the study. The digital divide existing in the population served in our unit may explain why some users did not feel comfortable with the electronic format and highlights the need to combine different strategies when presenting the survey that allows us to obtain responses from a greater number of users.
In general, we highlight the high percentage of positive responses to most of statements, with average scores >5, especially for those statements that evaluate the handling of information during the consultation and the perception of the quality of the visit. Thus, the overall assessment of the care received was 5.4 points out of 6. However, we must emphasize that there are issues related to space and time management worst rated and we must view this information as an opportunity to improve our service.
The waiting time prior to the consultation was reported as inadequate for almost half of the users, thus the corresponding statement was the lowest scored on the survey.
Similarly, 1-third of the users felt that the environment in which the consultation was carried out was inadequate. This result indicates the need to remodel this space, taking into consideration the opinion of the users to cocreate friendly environments where they feel comfortable and are guaranteed to have a good experience. Nine percent (9%) of respondents expressed disagreement with the role they have in decision-making. Although this is a small percentage, these data highlight the need to continue working on a patient-centered model where patients they perceive that their opinions are heard. There are few published data on satisfaction reported by PLHIV regarding the care they receive [18,19] in general, satisfaction is favorable, similar to our data, but the results are difficult to compare given that different PREMs were used.
In addition to the survey, we have used the NPS, a simple and validated tool that allows a quick and simple analysis of whether things are being done well in a job or service. Although this tool was developed 2 decades ago, [23] its use in the health field has been more recent and focuses on the benchmarking of private services. To date, there are no data in the literature on its use in the care of PLHIV. A NPS value above 50, such as that obtained in our study, is considered a high value and is synonymous with a positive score. Values similar to those obtained in this study, a NPS of 55, have been reported in studies conducted in the private health sector [25,26] of our setting. We are struck by the fact that Spanish respondents were more critical in their assessment and, therefore, significantly lowered the percentage of respondents classified as promoters. On the contrary, the responses of non-Spanish origin were more favorable.
We are aware that simply collecting data regarding the experience of patients is not sufficient, and that it is necessary to take action, as recommended by experts through PREMs. [27] The purpose of this study was to identify areas that need improvement, and we will evaluate NPS annually to verify whether any improvements impact the HRQoL perceived by the users.
This study has some limitations. Firstly, a bias for self-selection may have affected our results since only 1-third of the patients responded and their socio-demographic profile differed slightly from the average profile of the entire PLWHIV cohort who attended our hospital, as previously mentioned. Secondly, we don't have prior experience in large e-surveys sent to our patients with which compare the response rate. Finally, there is little published data on PREMS in PLWHIV to compare our findings to, and there is no data regarding NPS in the specific HIV infection field. We see this study as a starting point for further research in this area.
In conclusion, monitoring patient reported experience measures in PLWHIV actively receiving outpatient care in our hospital allowed to identify the users' perception on quality of the care received, to determine the rate of satisfaction with the care and to identify areas for improvement. The incorporation of the PREMs in the HIV units is essential to continue working on the improvement of the HRQoL of our patients. The periodic review of these tests will evaluate the effectiveness of the changes implemented after the needs detected have been addressed.